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Abstract. A symmetric tensor is a higher order generalization of a symmetric matrix. In 
this paper, we study various properties of symmetric tensors in relation to a decomposition into 
a symmetric sum of outer product of vectors. A rank-1 order-fc tensor is the outer product of k 
non-zero vectors. Any symmetric tensor can be decomposed into a linear combination of rank-1 
tensors, each of them being symmetric or not. The rank of a symmetric tensor is the minimal 
number of rank-1 tensors that is necessary to reconstruct it. The symmetric rank is obtained when 
the constituting rank-1 tensors are imposed to be themselves symmetric. It is shown that rank and 
symmetric rank are equal in a number of cases, and that they always exist in an algebraically closed 
field. We will discuss the notion of the generic symmetric rank, which, due to the work of Alexander 
and Hirschowitz, is now known for any values of dimension and order. We will also show that the 
set of symmetric tensors of symmetric rank at most r is not closed, unless r = 1. 

Key words. Tensors, multiway arrays, outer product decomposition, symmetric outer product 
decomposition, CANDECOMP, PARAFAC, tensor rank, symmetric rank, symmetric tensor rank, generic 
symmetric rank, maximal symmetric rank, quantics 
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1. Introduction. We will be interested in the decomposition of a symmetric 
tensor into a minimal linear combination of symmetric outer products of vectors (i.e. 
of the form v <g> v <£> ■ ■ ■ <g> v). We will see that a decomposition of the form 



always exists for any symmetric tensor A (over any field). One may regard this as a 
generalization of the eigenvalue decomposition for symmetric matrices to higher order 
symmetric tensors. In particular, this will allow us to define a notion of symmetric 
tensor rank (as the minimal r over all such decompositions) that reduces to the matrix 
rank for order-2 symmetric tensors. 

We will call the symmetric outer product decomposition of the symmet- 

ric tensor A and we will establish its existence in Proposition 14.21 This is often 
abbreviated as CanD in signal processing. The decomposition of a tensor into an 
(asymmetric) outer product of vectors and the corresponding notion of tensor rank 
was first introduced and studied by Frank L. Hitchcock in 1927 [35] ISO]- This same 
decomposition was rediscovered in the 1970s by psychomctricians in their attempts to 
define data analytic models that generalize factor analysis to multiway data [59] . The 
name CANDECOMP, for 'canonical decomposition', was used by Carrol and Chang [11] 
while the name PARAFAC, for 'parallel factor analysis', was used by Harshman [28] for 
their respective models. 

The symmetric outer product decomposition is particularly important in the pro- 
cess of blind identification of under- determined mixtures (UDM), i.e. linear mixtures 
with more inputs than observable outputs. We refer the reader to [14 ] 1X7 ] [20 ] [49 ] [50] 
and references therein for a list of other application areas, including speech, mobile 
communications, machine learning, factor analysis of fc-way arrays, biomedical engi- 
neering, psychomctrics, and chemometrics. 
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t Department of Computer Science and the Institute for Computational and Mathematical Engi- 
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Despite a growing interest in the symmetric decomposition of symmetric tensors, 
this topic has not been adequately addressed in the general literature, and even less so 
in the engineering literature. For several years, the alternating least squares algorithm 
has been used to fit data arrays to a multilinear model 36, 50J. Yet, the minimization 
of this matching error is an ill-posed problem in general, since the set of symmetric 
tensors of symmetric rank not more than r is not closed, unless r = 1 (see Sections 
[6] and [8]) — a fact that parallels the illposcdness discussed in [21] . The focus of this 
paper is mainly on symmetric tensors. The asymmetric case will be addressed in a 
companion paper, and will use similar tools borrowed from algebraic geometry. 

Symmetric tensors form a singularly important class of tensors. Examples where 
these arise include higher order derivatives of smooth functions |40j , and moments and 
cumulants of random vectors [43] . The decomposition of such symmetric tensors into 
simpler ones, as in the symmetric outer product decomposition, plays an important 
role in independent component analysis [14] and constitutes a problem of interest 
in its own right. On the other hand the asymmetric version of the outer product 
decomposition defined in (|4.ip is central to multiway factor analysis [50j . 

In Sections [2] and [3] we discuss some classical results in multilinear algebra [5l[26l 
139] [42] HU [63] and algebraic geometry [27] [64] . While these background materials are 
well-known to many pure mathematicians, we found that practitioners and applied 
mathematicians (in signal processing, neuroimaging, numerical analysis, optimization, 
etc) — for whom this paper is intended — are often unaware of these classical results. 
For instance, some do not realize that the classical definition of a symmetric tensor 
given in Definition 13.21 is equivalent to the requirement that the coordinate array 
representing the tensor be invariant under all permutations of indices, as in Definition 
13.11 Many authors have persistently mislabeled the latter a ' super ■symmetric tensor' 
(cf. [iniGHlIll])- In fact, we have found that even the classical definition of a symmetric 
tensor is not as well-known as it should be. We see this as an indication of the need to 
inform our target readership. It is our hope that the background materials presented 
in Sections [2] and [3] will serve such a purpose. 

Our contributions will only begin in Section [¥] where the notions of maximal and 
generic rank are analyzed. The concepts of symmetry and genericity are recalled in 
Sections [3] and [4] respectively. The distinction between symmetric rank and rank is 
made in Section[4j and it is shown in Section[5]that they must be equal in specific cases. 
It is also pointed out in Section[6]that the generic rank always exists in an algebraically 
closed field, and that it is not maximal except in the binary case. More precisely, the 
sequence of sets of symmetric tensors of symmetric rank r increases with r (in the 
sense of inclusion) up to the generic symmetric rank, and decreases thereafter. In 
addition, the set of symmetric tensors of symmetric rank at most r and order d > 2 is 
closed only for r = 1 and r = R$ , the maximal symmetric rank. Values of the generic 
symmetric rank and the uniqueness of the symmetric outer product decomposition 
are addressed in Section [7] In Section [5J we give several examples of sequences of 
symmetric tensors converging to limits having strictly higher symmetric ranks. We 
also give an explicit example of a symmetric tensor whose values of symmetric rank 
over M. and over C are different. 

In this paper, we restrict our attention mostly to decompositions over the complex 
field. A corresponding study over the real field will require techniques rather different 
from those introduced here, as we will elaborate in Section [821 

2. Arrays and tensors. A k-way array of complex numbers will be written 
in the form A = lo,j 1 ...j k l^""'" k k =1 , where aj 1 ...j k G C is the (ji, . . . , jfc)-cntry of 
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the array. This is sometimes also called a fc-dimensional hypermatrix. We denote 
the set of all such arrays by C niX "' xn '% which is evidently a complex vector space 
of dimension ni • ■ ■ Hfc with respect to entry-wise addition and scalar multiplication. 
When there is no confusion, we will leave out the range of the indices and simply 
write A = {a n ... 3k \ € C" 1 *-*"*. 

Unless noted otherwise, arrays with at least two indices will be denoted in up- 
percase; vectors are one-way arrays, and will be denoted in bold lowercase. For our 
purpose, only a few notations related to arrays [HJ HO] are necessary. 

The outer product (or Segre outer product) of k vectors u £ C" 1 , v € C™ 2 , . . . , z £ 
C lk is defined as 

u ® v <g> • • ■ ® z : = {u n v n ■ ■ ■ zjj"^::;;^* !• 

More generally, the outer product of two arrays A and B, respectively of orders k and 
£, is an array of order k + £, C = A® B with entries 

c ii---ikji---je := a ii---ih^ji--jf 

For example, the outer product of two vectors, u<8> v, is a matrix. The outer product 
of three vectors, or of a matrix with a vector, is a 3-way array. 

How is an array related to a tensor? Recall that a tensor is simply an element in 
the tensor product of vector spaces [3 [26, 39, 42, (44] [63]. One may easily check that 
the so-called Segre map 

if : C" 1 x ■•• xC l -> C" lX - xnfc , 
(u, . . . , z) l-> u g> ■ ■ • ® z 

is multilinear. By the universal property of the tensor product [51 121)1 13"9l l4"2l [HI I63j . 

there exists a linear map 

C™ 1 <g> • ■ ■ <g> C lk 




Since dim(C ni <8>- • •<g)C" fe ) = dim(C™ lX ' " x " fc ), is an isomorphism of the vector spaces 
C™ 1 <g> • • • ® C ,lfc and C" lX '-- x " fc . Consider the canonical basis of C™ 1 <g> • • • ® C" fc , 

{eV ® ■ • • ® ej-J 5 | 1 < ji < m, .. . , 1 < j fc < n fe }, 

where {e^ , . . . , e^} denotes the canonical basis in C ne , £ = 1, . . . , k. Then 6 may be 
described by 

"(>:"::;; ; ^.-^^ ® - ® *%) - i^m::^ 

So an order-fc tensor in C™ 1 ® • • • ® C™ fc and a fc-way array in C™ 1 x ' " x nk that represents 
the tensor with respect to a basis may be regarded as synonymous (up to, of course, 
the choice of basis). We will illustrate how the /c-array representation of an ordcr-/c 
tensor is affected by a change-of-basis. Let A = [a yfe ] G C" lX ™ 2X " 3 and let L, M, 
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and N be three matrices of size r\ x ni, r 2 x n 2 , and r 3 x n 3 , respectively. Then 
the tensor A may be transformed by the multilinear map (L, M, N) into a tensor 
-4' = Wpqrl 6 C riXr2Xr3 defined by 

a P9'' = / J. ■ ,. lpi m qj n rkaijk- (2.1) 

When n = rii and L, M , A" are nonsingular matrices, the above multilinear map may 
be thought of as a changc-of-bascs (refer to [21] for further discussions). Wc will call 
this map a multilinear transform of A. 

In addition to the outer product, we also have an inner product or contraction 
product of two arrays. The mode-p inner product between two arrays A, B having 
the same pih dimension is denoted A* p B, and is obtained by summing over the pih 
index. More precisely, if A and B are of orders k and I respectively, this yields for 
p = 1 the array C = AuiB of order k + £ — 2: 

c i 2 ---ikj 2 —h ~ a>ai 2 —i k b a j 2 ...j e . 

Note that some authors [20] [24[ [59] denoted this contraction product as A x p B or 
(A,B) p . By convention, when the contraction is between a tensor and a matrix, it 
is convenient to assume that the summation is always done on the second matrix 
index. For instance, the multilinear transform in (|2.1|) may be expressed as A' = 
A»\L»2M»zN . An alternative notation for (|2.ip from the theory of group actions is 
A 1 = (L, M, N) ■ A, which may be viewed as multiplying A on 'three sides' by the 
matrices L, M, and N [2l~ll32] . 

3. Symmetric arrays and symmetric tensors. We shall say that a fc-way 
array is cubical if all its k dimensions are identical, i.e. ni = • ■ ■ = Hfe = n. A cubical 
array will be called symmetric if its entries do not change under any permutation 
of its k indices. Formally, if denotes the symmetric group of permutations on 
{1, . . . , k}, then we have 

Definition 3.1. A k-way array [a^...^] <G C nx ' " xn is called symmetric if 

a i a {l)---icr(k) ~ a il---ik ' • • • i S {1, . . . , Tl}, 

for all permutations a G 

For example, a 3-way array [a^fe] € C xnx ' 1 is symmetric if 

— ®>ikj — Cljik — CLjki — &kij — Ctkji 

for all fee {1, • ■ ■ ,n}. 

Such arrays have been improperly labeled 'supersymmetric' tensors (cf. [TTJ] (M[ 
|45] among others); this terminology should be avoided since it refers to an entirely 
different class of tensors [7]. The word 'supersymmetric' has always been used in 
both mathematics and physics [25l [60l [62] to describe objects with a ^-grading and 
so using it in the sense of [101 EH HS] is both inconsistent and confusing (the correct 
usage will be one in the sense of [7]). In fact, we will show below in Proposition ^. 7l that 
there is no difference between Definition 13.11 and the usual definition of a symmetric 
tensor in mathematics [3 [26l [39j [42] [44l [63]. In other words, the prefix 'super' in 
'supersymmetric tensor', when used in the sense of [10] . l3"4" l [45] . is superfluous. 

We will write T k (C n ) := C" ® ■ ■ • ® C" (k copies) , the set of all order-fc dimension- 
n cubical tensors. We define a group action &k on T fe (C™) as follows. For any a € &k 
and Xi x ® ••• ®Xi k e T fe (C n ), wc let 

o-fxi! <g> ■ • • <g> x;J := Xj ® ■ ■ • <g> Xi^ . 
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and extend this linearly to all of T fc (C n ). Thus each a G &k defines a linear operator 
a : T fc (C n ) — > T fc (C"). The standard definition of a symmetric tensor in mathematics 
[51 SH (SI (S3] looks somewhat different from Definition 13.11 and is given as 

follows. 

Definition 3.2. An order-k tensor A G T fc (C") is symmetric if 

a{A)=A (3.1) 

for all permutations a G ©fe. TTie set of symmetric tensors in T fe (C") will be denoted 
by S fc (C"). 

Let S : T fe (C n ) — > T fc (C") be the linear operator defined by 

fe! ^cree fc 

Note that given any cG 6t, 

croS' = S'o(T = S'. 

Here o denotes the composition of the linear operators a and S. 

Proposition 3.3. An order-k tensor A € T fc (C") is symmetric if and only if 

k\ * — 'o-e©*; 
Proof. Clearly, if A is symmetric, then 

S ( A ) = ItE ,r ^^E ,r A = A 
fe! ^^cre&k fe! ^-^o-ebfc 

Conversely, if S(A) = A, then 

= a(S(A)) =ao S(A) = S(A) = A 

for all a G and so A is symmetric. □ 

In other words, a symmetric tensor is an eigenvector of the linear operator S with 
eigenvalue 1. S fc (C") is the 1-eigenspace of S : T k (C n ) -> T fc (C"). Proposition EH 
implies that S fc (C") = S(T k (C")) and it is also easy to see that S is a projection of 
T fc (C") onto the subspace S fc (C"), i.e. S 2 = S. 

3.1. Equivalence with homogeneous polynomials. We adopt the following 
standard shorthand. For any e^ , . . . , e% k G C" with i\, . . . , ih G {1, . . . , n}, we write 

e h ■■■e ik := S(e h <g> • • • Cg> e ife ) = — E (TeSfc e MD ® ' " ® e M *r ( 3 - 2 ) 

Then since ^cr = S 1 , the term • • • e.; fc depends only on the number of times each e^ 
enters this product and we may write 

e il ' ' ' e ifc — e l ' ' ' e n" (3-3) 

where pi is the multiplicity (which may be 0) of occurrence of ej in e^ • • • e !fc . Note 
that pi, . . . ,p n are nonnegative integers satisfying p\ + ■ ■ ■ + p n = k. 
Proposition 3.4. Let {e l7 . . . ,e„} be a basis o/C". Then 

{S(e h <g> • ■ ■ ® e; J | 1 < ix < ■ ■ ■ < i k < n} 
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or, explicitly, 

{ k\ 5^66* ® ' ' " ® ^(W | 1 < *1 < ' ' ' < ^ < 

is a feasis o/S (C n ). Furthermore, 

dim c S fe (C") = + 

Proof. Since B = {e^ ® ■ ■ • ® e, fe | 1 < £x < n, . . . , 1 < ik < ti} is a basis for 
T fe (C") and since 5 maps T fe (C n ) onto S fc (C"), the set 

5(6) = {e n •• -e ik | 1 < h < ■■ ■ < i k < n} = {ef • • -eg" | pi + • • ■ +p„ = fc} 

spans S fe (C n ). Vectors in 5(B) are linearly independent: if (px, . . . ,p„) ^ (gi, . . . , q n ), 
then the tensors e^ 1 • • • e^" and 1 • • • e^" are respectively linear combinations of 
two non-intersecting subsets of basis elements of T fe (C n ). The cardinality of 5(B) is 
precisely number of partitions of k into a sum of n nonnegative integers, i.e. 
□ 

If we regard ej in (|3.3[) as variables (i.e. indeterminates) , then every symmetric 
tensor of order k and dimension n may be uniquely associated with a homogeneous 
polynomial of degree k in n variables. Recall that these arc just polynomials in n vari- 
ables whose constituting monomials all have the same total degree k. Homogeneous 
polynomials are also called quantics and those of degrees 1,2, and 3 are often called 
linear forms, quadratic forms, and cubic forms (or just cubics) respectively. From 
now on, we will use more standard notation for the variables — Xj instead of e^-. So 
the monomial on the RHS of (|3 . 3[) now becomes x^ 1 ■ ■ ■ x^ . To further simplify this 
notation, we will adopt the following standard multi-index notations: 



-w—w-n » — . n 



Pk, 



where p denotes a fc-vector of nonnegative integers. We will also write <C[xi, . . . , x n ]k 
for the set of homogeneous polynomials of degree k in n variables (again a standard 
notation). Then any symmetric tensor [aj!--.^]] = [oj] G S fe (C") can be associated 
with a unique homogeneous polynomial F G C[xi, . . . , x n ]k via the expression 

F(x)=^.a iX ^), (3.4) 

where for every j = (ji, ■ ■ ■ ,jk) } one associates bijectively the nonnegative integer 
vector p(j) = {pi(j), ■ ■ ■ ,p n (j)) with Pj(j) counting the number of times index j 
appears in j [161114] . We have in particular \p(j)\ — k. The converse is true as well, 
and the correspondence between symmetric tensors and homogeneous polynomials is 
obviously bijective. Thus 

S k (C n )^C[x u ...,x n ] k . (3.5) 

This justifies the use of the Zariski topology, where the elementary closed subsets 
are the common zeros of a finite number of homogeneous polynomials [48] . Note that 
for asymmetric tensors, the same association is not possible (although they can still 
be associated with polynomials via another bijection). As will be subsequently seen, 
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this identification of symmetric tensors with homogeneous polynomials will allow us 
to prove some interesting facts about symmetric tensor rank. 

We will now proceed to define a useful 'inner product' on C[xi, . . . , x n ]k- For any 
F, G £ C[xi, . . . , x n ]k written as 

we let 

Note that (■, •) cannot be an inner product in the usual sense since (F, F) is in general 
complex valued (recall that for an inner product, we will need (F, F) > for all F). 
However, we will show that it is a non-degenerate symmetric bilinear form. 

Lemma 3.5. The bilinear form (•, •) : C[xi, . . . , x n ]k X C[xi, . . . , x n ]k — ► C defined 
above is symmetric and non-degenerate. In other words, (F, G) = (G, F) for every 
F,G G C[xi, . . . , x n ]k; and if (F, G) = for all G £ C[x 1: . . . , x n ]k, then F = 0. 

Proof. The bilincarity and symmetry is immediate from definition. Suppose 
(F, G) = for all G € C[xi, . . . , x n ]fe. Choose G to be the monomials 



G p (x) 



Pi,-- - ,Pn 



where |p| = fe and we see immediately that 

a p = (F, G p ) = 0. 

Thus F = 0. □ 

In the special case where G is the fcth power of a linear form, we have the fol- 
lowing lemma. The main interest in introducing this inner product lies precisely in 
establishing this lemma. 

Lemma 3.6. Let G = (0iXi + • ■ ■ + (3 n x n ) . Then for any F £ C[x±, . . . ,x n ]k, 
we have 

(F, G) = F([3i, . . . , /3 n ), 

i.e. F evaluated at (flx, ...,/?„) £ C". 

Proof. Let b p = /3f 1 ---f3f™ for all p = (jp\, . . . ,p n ) such that \p\ = k. The 
multinomial expansion then yields 

((3 lXl + ■ ■ ■ + /3 n x n ) k = V ( k )b p x.P. 

^\ P \=k \p u ... ,p n J 

For any F (x) = £ |p|=fe ( pi k Ja p ^, 

F^,...,^)^^^ " Ja pbp = (F,G) 



as required. □ 
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3.2. Equivalence with usual definition. As mentioned earlier, we will show 
that a tensor is symmetric in the sense of Definition 13. 21 if and only if its corresponding 
array is symmetric in the sense of Definition 13.11 

PROPOSITION 3.7. Let A £ T fe (C n ) and [ctjvjj £ C nx ' " xn be its corresponding 
k-array. Then 

a(A) = A 

for all permutations a £ &k if and only if 



i k £ {1, . . . ,n}, 



for all permutations a £ &k- 

Proof. Suppose [a^...^! £ C nx ' " xn is symmetric in the sense of Definition 13.11 
Then the corresponding tensor 

A= / 0,^... j fe ejj ® • • • ® e» fc , 

where {ei, . . . ,e n } denotes the canonical basis in C™, satisfies the following: 



8{A) = J2 



ii,...,»fc=l 



a il ...i k S{e il ®---®e ifc ) 



(5 linear) 



; 1 / / 1 / , e Mi) 

fc! ' — 'ti,...,«fc=l \A — 'o-SSfc v ' 

-Y n \y 

fc! ^— 'ti i*=i 

fc! ^ 



'ii,...,l fc =l 



fc!a il ... ijt ei 1 <g> • ■ ■ g> e 



(Ifflii-iJ symmetric) 



Hence A is a symmetric tensor in the sense of Definition [ 

Conversely, let A £ T fc (C") be symmetric in the sense of Definition 13.21 and 



A = J2 



ii ,...,i k = l 



be the expression of A with respect to {e^ 
canonical basis of T fc (C"). Then 



ei k I 1 <«!,... ik < "}> tnc 



5(A) = A 



implies 

EI 



fc! ^ 



fc! ^£61 



CI 



E 



ti,...,ifc=l 



) 67 ., . 



'u »*=1 

Since {e^ ® • • • ® ei fc | 1 < ii, . . . < n} is a linearly independent set, we must have 
fclX^ee*, ^ 1 ''''^ =a *i- i * for alHi,...,i fe e {!,..., 71}. (3.6) 
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For any given r G @fe, we have 



1 y 

fc! ^-^cre-i-efc 



= fcj ^^6,- a Mi)"M*) ( rSfc = 6fe as &k is a § rou P) 

= o il ...<, (by(|nU). 

Since this holds for arbitrary r G &k, the array [a^. ..$ fc ] is symmetric in the sense of 
Definition O □ 



4. Notions of rank for symmetric tensors. Wc will discuss two notions of 
rank for symmetric tensors — the outer product rank (defined for all tensors) and 
the symmetric outer product rank (defined only for symmetric tensors) . We will show 
that under certain conditions, they are one and the same. However it is not known if 
they are equal on all symmetric tensors in general. 

4.1. Outer product decomposition and rank. Any tensor can always be 
decomposed (possibly non-uniqucly) as: 



A = 



V ? u* (8 v< (8 • • • ® Wi. (4.1) 

^ — J i— 1 



The tensor rank, rank(A), is defined as the smallest integer r such that this decom- 
position holds exactly [2S1|3D]. Among other properties, note that this outer product 
decomposition remains valid in a ring, and that an outer product decomposition of 
a multilinear transform of A equals the multilinear transform of an outer product 
decomposition of A. In other words, if (|4.1|) is an outer product decomposition of A, 
then 

A»iL» 2 M» 3 ■ ■ ■ •k.N = Lui (8 Mvi (8 • • • <8 Aw^ 

^ — J i— 1 

is an outer product decomposition of A»iL»2M»3 • • ••feA, which may also be writ- 
ten as (L, M, . . . , A) • A. The outer product decomposition has often been regarded 
synonymously as the data analytic models CANDECOMP [TT] and PARAFAC [25] where 
the decomposition is used to analyze multiway psychometric data. 

Definition 4.1. The rank of A = [a^...^] G £dtx-xd k j s d e fl ne ^ as 

rank(^l) := min{r | A = X^=i u » ® v i ® ' ' " ® w 0- 

If A = [flj! - ^] G S (C n ), then we may also define the notion of symmetric rank via 

rank s (^4) := min{s | A = ^- =1 y, <8 • ■ ■ ® y,-}. 

Note that over C, the coefficients Xi appearing in decomposition (|1 . ip may be set 
to 1; this is legitimate since any complex number admits a fcth root in C. Henceforth, 
we will adopt the following notation 

k copies 

A 

y® fc :=y<g) ... <g)y. (4.2) 

If in (|4.ip . we have Uj = v, = • ■ • = Wj for every i, then we may call it a symmet- 
ric outer product decomposition, yielding a symmetric rank, ranks (A) . Constraints 
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other than full symmetry may be relevant in some application areas, such as partial 
symmetry as in INDSCAL [TTJ[57], or positivity/non-negativity [4*11 [5D1IM] . 

The definition of symmetric rank is not vacuous because of the following result. 

Lemma 4.2. Let A e S fc (C"). Then there exist y 1; . . . ,y s G C" such that 

Proof. What we actually have to prove, is that the vector space generated by the 
fcth powers of linear forms L(x) fe (for all L G C") is not included in a hyperplane of 
S fe (C"). This is indeed true, because otherwise there would exist a non-zero element 
of S fc (C™) which is orthogonal, under the bilinear form (•, •), to all L(x) fc for L G C™. 
Equivalently, by Lemma |3"1)I there exists a non-zero polynomial q(x) of degree k such 
that q(L) = for all L G C". But this is impossible, since a non-zero polynomial does 
not vanish identically on C™. □ 

Lemma 14.21 may be viewed as a particular case of a basic result in algebraic 
geometry, stating that the linear space generated by points of an algebraic variety 
that is not included in a hyperplane, i.e. a subspace of codimension 1, is the whole 
space [27l [18j [48]. For completeness, a proof of our special case is given above. Note 
that it follows from the proof that 

ranks^j^- 1 ) 

for all A G S fc (C"). 

On the other hand, given a symmetric tensor A, one can compute its outer product 
decomposition either in S fe (C") or in T k (C n ). Since the outer product decomposition 
in S fe (C") is constrained, it follows that for all A G S fe (C"), 

rank(A) < ranks (A). (4.3) 

We will show that equality holds gcncrically when ranks (-4) < n and when k is 
sufficiently large with respect to n, and always holds when ranks(A) = 1,2. While 
we do not know if the equality holds in general, we suspect that this is the case as we 
are unaware of any counterexample. 

4.2. Secant varieties of the Veronese variety. Let us recall here the corre- 
spondence between symmetric outer product decompositions and secant varieties of 
the Veronese variety. By the bijective correspondence between symmetric tensors and 
homogeneous polynomials established in (|3.5|) . we may discuss this in the context of 
homogeneous polynomials. The set of homogeneous polynomials that may be written 
as a /eth power of a linear form, f3(x) k = (/?iXi + - • ■+(3 n x n ) k for (3 = (/3i, . . . , (3 n ) G C™, 
is a closed algebraic set. We may consider this construction as a map from C" to the 
space of symmetric tensors given by 

v n , k :C" ^C[x 1 ,...,x n ] k ^S k (C n ), 
f3^f3(x) k - 

The image ^ n ,fc(C") is called the Veronese variety and is denoted V n ,k [27|, 164] . Fol- 
lowing this point of view, a symmetric tensor is of symmetric rank 1 if it corresponds 
to a point on the Veronese variety. A symmetric tensor is of symmetric rank r if it 
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is a linear combination of r symmetric tensors of symmetric rank 1 but not a linear 
combination of r — 1 or fewer such tensors. In other words, a symmetric tensor is of 
symmetric rank not more than r if it is in the linear space spanned by r points of the 
Veronese variety. The closure of the union of all linear spaces spanned by r points of 
the Veronese variety V n ,k is called^ the (r — l)th-secant variety of V n ,h- See [27l [64] 
for examples and general properties of these algebraic sets. In the asymmetric case, 
a corresponding notion is obtained by considering the Segre variety, i.e. the image of 
the Segre map defined in Section 

4.3. Why rank can exceed dimension. We are now in a position to state 
and prove the following proposition, which is related to a classical result in algebraic 
geometry stating that r points in C n form the solution set of polynomial equations 
of degree < r [27l pp. 6]. This implies that we can find a polynomial of degree 
< r — 1 that vanishes at r — 1 of the points Li but not at the last one, and hence 
the independence of polynomials . . . , L£ _1 follows. Since this proposition is 

important to our discussion in Section [5] (via its corollary below), we give a direct and 
simple proof below. 

Proposition 4.3. Let L\, . . . , L r £ C[xi, . . . , x n ]i, i.e. linear forms in n vari- 
ables. If for all i =/= j, Li is not a scalar multiple of Lj, then for any k > r — 1, the 
polynomials L\ , . . . ,L^ are linearly independent in C[xi, . . . , x n ]. 

Proof. Let fe > i — 1. Suppose that for some Ai, . . . , A r , X)I=i ^i-^f = 0- Hence, 
by the duality property of Lemma 13.61 

* — J l— 1 * — 

for all F £ <C[xi, . . . , x„]fc. Let us prove that we can find a homogeneous polynomial 
F of degree k that vanishes at L\, . . . , L r _\ and not at L r . 

Consider a homogeneous polynomial F of degree k > r — 1 that is a multiple 
of the product of r — 1 linear forms Hi vanishing at Li but not at L r . We have 
F(L r ) but F(Lj) = 0, 1 < j < 1 — 1. As a consequence, we must have A r = 0. 
By a similar argument, we may show that A^ = for all i = 1, . . . , r. It follows that 
the polynomials L*, . . . , are linearly independent. □ 

Notice that the bound r — 1 on the degree can be reduced by d if a d-dimensional 
linear space containing any d + 1 of these points does not contain one of the other 
points [271 pp. 6]red. In this case, we can replace the product of d + 1 linear forms 
Hi vanishing at d + 1 points by just 1 linear form vanishing at these d + 1 points. 

Corollary 4.4. Let vi, . . . ,v r € C" be r pairwise linearly independent vectors. 
For any integer k > r — 1, the rank-l symmetric tensors 

vf fe ,...,vf fc g S fc (C") 

are linearly independent. 

This corollary extends results of [TH1 Lemma 2.2, pp. 2] and [331 Appendix]. Note 
that vectors vi, . . . , vy need not be linearly independent. 

Example 4.5. Vectors vi = (1, 0), V2 = (0, 1), andw^ = (1, 1), are pairwise non- 
collinear but linearly dependent. According to Corollary \4-4\ the symmetric tensors 



1 This seemingly odd choice, i.e. r — 1 instead of r, is standard 1271 1641 . The reason being that 
one wants to be consistent with the usual meaning of a secant, i.e. 1-secant, as a line intersecting 
two points in the variety. 
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v® , v® , vf are linearly independent for any k > 2. Evidently, we see that this 
holds true for k — 2 since the matrix below has rank 3: 



4.4. Genericity. Roughly speaking, a property is referred to as typical if it 
holds true on a non-zero-volume set and generic if is true almost everywhere. Proper 
definitions will follow later in Section [5] It is important to distinguish between typical 
and generic properties; for instance, as will be subsequently seen, there can be several 
typical ranks, but by definition only a single generic rank. We will see that there can 
be only one typical rank over C, and is thus generic. 

Through the bijection (|3.4|) . the symmetric outer product decomposition (|4.ip of 
symmetric tensors can be carried over to qualities, as pointed out in [16] . The bijection 
allows one to talk indifferently about the symmetric outer product decomposition of 
ordcr-fc symmetric tensors and the decomposition of degrec-/c quantics into a sum of 
linear forms raised to the fcth power. 

For a long time, it was believed that there was no explicit expression for the 
generic rank. As Rcznick pointed out in [47) . Clebsh proved that even when the 
numbers of free parameters are the same on both sides of the symmetric outer product 
decomposition, the generic rank may not be equal to ij( n+ fc -1 )- For example, in the 
case (k,n) = (4,3), there are m = 15 degrees of freedom but the generic symmetric 
rank i? s (4, 3) = 6 ^ 5 = ±(4). In fact, this holds true over both R [47] and C [22]. 
In Section [7] we will see that the generic rank in S fc (C") is now known for any order 
and dimension due to the ground breaking work of Alexander and Hirschowitz. 

The special case of cubics (k = 3) is much better known — a complete classifica- 
tion is known since 1964 though a constructive algorithm to compute the symmetric 
outer product decomposition has only been proposed recently [35] . The simplest case 
of binary quantics (n = 2) has also been known for more than two decades [6T | ri6 ] [38] 
— a result that is used in real world engineering problems [15] , 

5. Rank and symmetric rank. Let R$(k,n) be the generic symmetric rank 
and Rs(k,n) be the maximally attainable symmetric rank in the space of symmetric 
tensors S fe (C"). Similarly, let R(k,n) be the generic rank and R(k,n) be the max- 
imally attainable rank in the space of order- k dimension- n cubical tensors T fc (C"). 
Since S k (C n ) is a subspace of T fe (C"), generic and maximal ranks (when they exist) 
are related for every fixed order k and dimension n as follows: 



It may seem odd that the inequalities in (j5.1[) and (]4.3p are reversed, but there is no 
contradiction since the spaces are not the same. 

It is then legitimate to ask oneself whether the symmetric rank and the rank are 
always equal. We show that this holds generically when ranks (A) < n (Proposition 
I5.3P or when the order k is sufficiently large relative to the dimension n (Proposition 
I5.4p . This always holds (not just generically) when ranks(A) = 1, 2 (Proposition 15. 5D . 
We will need some preliminary results in proving these assertions. 

Lemma 5.1. Let yi , . . . , y s G C n be linearly independent. Then the symmetric 
tensor defined by 



1 



1 1 





1 

1 1 



R{k,n) > Rs(k,n), and R(k,n) > Rs(k,n). 



(5.1) 
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has rankg (A) = s - 

Proof. Suppose ranks (A) = r - Then there exist z 1; . . . , z r G C™ such that 

yf k =A = J2 r zf. (5.2) 

* — 'z— 1 * — ^ J — 1 ■> 

By the linear independence of yi, . . . ,y s , there exist covectors ip\ , . . . , tp s G (C n )* 
that are dual to yi, . . . , y s , i.e. 

J 1 if i = j, 
(0 it it 3- 

Contracting both sides of jO]) in the first k - 1 modes with y>f (fe_1) G S fc_1 ((C n )*), 
we get 

Yi = X ; ;"-' z '- 

where ay = yi(zj) fe_1 . In other words, yi G spanjzi, . . . , z r }. Since this holds for 
each i = 1, . . . ,s, it implies that the s linearly independent vectors yi, . . . , y s are 
contained in spanjzi, . . . , z r }. Hence we must have r > s. On the other hand, it is 
clear that r < s. Thus we must have equality. □ 

Lemma 5.2. Let s < n. Let A G S fc (C") with ranks(A) = s and 

— 'i— i 

be a symmetric outer product decomposition of A. Then vectors of the set {yi, . . . ,y s } 
are generically linearly independent. 
Proof. We will write 

y s := {A G S fc (C") | ranks (A) < s} and Z s := {A G S fc (<C") | rank s (A) = s}. 

Define the map from the space of n x s matrices to order- k symmetric tensors, 

/ : C" xs -> S fe (C"), 

[ yi ,...,y s ]^^ iy f fc . 

It is clear that / takes C" xs onto y s (i.e. f(C nxs ) = y s ). We let E and E x be the 
subsets of rank-deficient and full-rank matrices in C™ xs respectively. So we have the 
disjoint union 

^oU^i = C" XS , E Q DE 1 =0. 

Recall that the full-rank matrices are generic in C™ xs . Recall also that Eq is an 
algebraic set in <C nxs defined by the vanishing of all s x s principal minors. By the 
previous lemma, f(E\) C Z s . The set of symmetric tensors, 

EL, yf 

in Z s for which {yi, . . . , y s } is linearly dependent, i.e. [yi, . . . , y s ] is rank deficient, is 
simply 

Z.nf(Ea). 
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Since / is a polynomial map and E is a non-trivial algebraic set, we conclude that 
f(Ei) is generic in Z s . □ 

PROPOSITION 5.3. Let A e S k (C n ). 7/rank s (A) < n, then rank(A) = rank s (A) 
generically. 

Proof. Let r = rank(A) and s — ranks (A). So there exist decompositions 

E>f«-"-^f =^ = K^f- (5-3) 

By Lemma 15 .21 we may assume that for a generic A £ Z s , the vectors yi , . . . , y s are 
linearly independent. As in the proof of Lemma |5. 11 we may find a set of covectors 
ipi, . . . ,ip s <G (C n )* that are dual to y l7 . . . , y s , i.e. 



1 if i = j, 
if i ^ j. 



Contracting both sides of (ITS) in the first k - 1 modes with <pf {k 1} e S fe - 1 ((C")*), 
we get 



\ - 



« - v 



where aij = ifi (x^ ) • • • ifi (xj fc j = l,...,r. Since this holds for each i = 
1, . . . , s, it implies that the s linearly independent vectors yi, . . . , y s are contained 
in spanjxj , . . . , x£ }. Hence we must have r > s. On the other hand, it is clear 
that r < s. Thus we must have equality. □ 

We will see below that we could have rank(A) = ranks (A) even when the con- 
stituting vectors yi, . . . ,y s are not linearly independent. The authors would like to 
thank David Gross for his help in correcting an error in the original proof. 

Proposition 5.4. Let yi, . . . ,y s £ C™ be pairwise linearly independent. If k is 
sufficiently large, then the symmetric tensor defined by 



A := 



satisfies rank(A) = ranks (^4) generically. 

Proof. Let r = rank(A) and s = ranks (A). So there exist decompositions 

E^xf^-^xfW^^yf. (5.4) 

Note that the lhs may be written 2~,i=i Yi ® Yi i where we have assumed, 
without loss of generality, that k is even. By Proposition 14. 3[ when k is suffi- 
ciently large, the order-(fc/2) tensors yf°^ 2 , . . . ,yf k ^ 2 are generically linearly inde- 
pendent. Hence we may find functional $i, ...,<& a G S fe / 2 (C n )* that are dual to 
yf fc/2 ,...,y? fc/2 GS fc /2(C»),i.c. 

Contracting both sides of (|5.4[) in the first k/2 modes with $i, we get 

E r =1 ^? /2+1) ®---®4 fe) =yf /2 : 
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where c&ij = <&i (x^ 1 ' <g • • • (g x^ 2 -* ), j = 1 , . . . , r. Since this holds for each i = 1, . . . , s, 
it implies that the s linearly independent vectors y 1 , . . . , y s are contained in 
span{x^ 2+1 - ) ® • • • ® x^ , . . • , Xr ® • • • <g x^ }. Hence we must have r > s. On 
the other hand, it is clear that r < s. Thus we must have equality. □ 

PROPOSITION 5.5. Let A e S fc (C"). 7/rank s (A) = 1 or 2, then rank(A) = 
ranks (^4). 

Proof. If ranks (A) = 1, then rank(A) = 1 clearly. If ranks (A) = 2, then 

A = yf k +yf k 

for some yi,y2 £ C". It is clear that yi and y2 must be linearly independent or 
otherwise y2 = ayi implies that 

A=(/3 yi )® fc 

for any (3 = (1 + a' ) 1 ^, contradicting ranks(A) = 2. It follows from the argument in 
the proof of Proposition 15.31 with s = 2 that rank(A) = 2. □ 
The following result will be useful later. 

Proposition 5.6. Let vi and V2 be two linearly independent vectors in C". 
Then for any k > 1, the following order-k symmetric tensor: 



vi <g> v 2 ® v 2 (X) • • • ® v 2 + v 2 (8> vi (g) v 2 ® • • • (8> v 2 

+ v 2 (g) v 2 (g) vi (g) • • • (g) v 2 H h v 2 (g v 2 (g v 2 (g • • • (g vi (5.5) 

is of symmetric rank k. 

Proof. It is not hard to check that the symmetric tensor in ()5.5(1 is associated with 
the quantic p(zi,z 2 ) = ziz^ 1 , up to a constant multiplicative factor (where zi,z 2 
are the first two coordinate variables in [zi, . . . , z n )). 

To prove that this quantic is of symmetric rank fc, we are going to show that 
p(zi, Z2) can be decomposed into a sum of powers of linear forms as 

p(z 1 ,z 2 ) = Y] k \i{a lZl + (3iZ 2 ) k . (5.6) 

* ^2 — 1 

There arc infinitely many possibilities of choosing coefficients (ati,/3i) but we just need 
to provide one solution. Take a.\ = ■ ■ ■ = a r = 1 and fii, . . . , fik distinct such that 

, a = °- ( 5J ) 

* — 

First we express all quantics in terms of the canonical basis scaled by the binomial 
coefficients: 

{z k ,kz k - 1 z 2 ,...,kz 1 z k -\z k }. 

In this basis, the monomial kz\z\~ x can be represented by a (fc + l)-dimensional vector 
containing only one non-zero entry. The quantic {zi + f3iZ 2 ) k is then represented by 
the vector 

[l,(3 i ,(3l...,tf}eC k+1 . 
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The existence of coefficients Ai, . . . , A& such that we have the decomposition (|5.6p is 
equivalent to the vanishing of the (fc + 1) x (fc + 1) determinant 





1 0i 

1 0k 



PI 



1 

fc-i 



PI 



fe-i 





0\ 



01 



(5.8) 



,0k) 
,0k- 



An explicit computation shows that this determinant is ±Q3j=i 0i)Vk{0i, 
where Vk{0\, ■ ■ ■ ,0k) is the Vandermonde determinant of degree fc — 1 of 0i, 
Thus by (|5.7p . the determinant in (|5.8[) vanishes. 

This proves that the symmetric rank of z\z\ is — Note that the symmetric 
rank cannot be smaller than k because removing any row of the matrix of (|5.8[) still 
yields a matrix of rank fc, if the 0i are distinct (see also Proposition 14.3ft . □ 

This proof is constructive, and gives an algorithm to compute a symmetric outer 
product decomposition of any binary symmetric tensor of the form (|5.5| . For example, 
the reader can check out that the decompositions below may be obtained this way. 

Example 5.7. The quantics A8zfz2 and 60z 4 ;z 2 are associated with the symmetric 
tensors of maximal rank A31 and A±\ respectively. Their symmetric outer product 
decompositions are given by 

A 31 = 8( Vl + v 2 )® 4 - 8(V! - v 2 )® 4 - ( Vl + 2v 2 ) m + ( Vl - 2v 2 )® 4 , 

An = 8( Vl + v 2 )® 5 - 8(vi - v 2 )® 5 - (vi + 2v 2 )® 5 + ( Vl - 2v 2 )^ 5 + 48vf 5 . 

The maximal symmetric rank achievable by symmetric tensors of order k and 
dimension n = 2 is fc, i.e. Rs(k,2) = fc. One can say that such symmetric tensors 
lie on a tangent line to the Veronese variety of symmetric rank-1 tensors. In |13j . an 
algorithm has been proposed to decompose binary forms when their rank is not larger 
than fc/2; however, this algorithm would not have found the decompositions above 
since the symmetric ranks of A31 and A41 exceed 4/2 and 5/2 respectively. 



6. Generic symmetric rank and typical symmetric ranks. 

and dimension, define the following subsets of symmetric tensors y r :■ 



For given order 

= {A e S fc (C") I 



ranks(A) < r} and Z r := {A € S (C n ) | ranks (A) = r}. Also, denote the correspond- 
ing Zariski closures by 3-V and Z r respectively. Recall that the Zariski closure [18] of 
a set S is simply the smallest variety containing S. For every r £ N, we clearly have 



y r -i U Z r = y r and J 7 ! 



r copies 

The quantities Rs(k, n) and Rs(k, n) may now be formally defined by 

R s (k, n) := min{r | y r = S k (C n )} and R s (k, n) := min{r | y r = S fe (C")}. 

By definition, we have Rs(k,n) < Rs(k,n). We shall prove in this section that a 
generic symmetric rank always exists in S fc (C"), i.e. there is an r such that Z r = 
S k (C n ), and that it is equal to Rs(k,n), thus justifying our naming Rs(k,n) the 
generic symmetric rank in Section [5] 

An integer r is not a typical rank if Z r has zero volume, which means that Z r 
is contained in a non-trivial closed set. This definition is somewhat unsatisfactory 
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since any mention of 'volume' necessarily involves a choice of measure, which is really 
irrelevant here. A better definition is as follows. 

Definition 6.1. An integer r is a typical rank if Z r is dense with the Zariski 
topology, i.e. if Z r = S fe (C n ). When a typical rank is unique, it may be called generic. 

We used the wording "typical" in agreement with previous terminologies [9l 1551 
I57j . Since two dense algebraic sets always intersect over C, there can only be one 
typical rank over C, and hence is generic. In the remainder of this section, we will 
write Rs = Rs{k, n) and R$ = Rs{k, n). We can then prove the following. 

Proposition 6.2. The varieties Z r can be ordered by inclusion as follows. If 

T\ < r 2 < Rs < r 3 < Rs, 

then 

Z ri C_ Z r - 2 C_ Z-^^ 2 Z T3 . 

Before proving this proposition, we first state two preliminary results. Recall that 
an algebraic variety is irreducible if it cannot be decomposed as the union of proper 
subvarieties (cf. [UJ PP- 51] and [HI pp. 34]). In algebraic geometry, it is known that 
the secant varieties of any irreducible variety are irreducible. Nevertheless, we will 
give a short proof of the following lemma for the sake of completeness. 

Lemma 6.3. The sets y r , r > 1, are irreducible algebraic varieties. 

Proof. For r > 1, the variety y r is the closure of the image y r of the map 

cpr :C™ xr ->S fc (C"), 
[ Ul ,...,u r ]^]T r uf k . 

Consider now two polynomials f,g such that fg = on y r . As y r is the Zariski 
closure of y r , this is equivalent to fg = on y r or 

(fg) ° tfr = (/ ° ip r )(g o ip r ) = 0. 

Thus either / = or g = on y r or cquivalently on y r , which proves that y r is an 
irreducible variety. For more details on properties of parameterized varieties, see |18j . 
See also the proof of [511 13 for third order tensors. □ 
Lemma 6.4. We have Rs = min{r | y r = y r +i}- 

Proof. Suppose that there exists r < Rs such that y r = y r +i- Then since 
y r Qy r +yi Q y r +i = y r , we have 

y r = y r + yi = y r + yi + yi = ■ ■ ■ = y r + yi + ■ ■ ■ + yi. 

As the sum of Rs copies of 3^1 is S fc (C"), we deduce that y r = S fc (C") and thus 
r > Rs, which contradicts our hypothesis. By definition, y^ s = y^ s+1 = S k (C n ), 
which proves the lemma. See also the proof of |51j for the asymmetric case. □ 
We are now in a position to prove Proposition 16.21 
Proof of Proposition^^ By Lemma 16.41 we deduce that for r < Rs, 

y r ^y r +i- 

As y r is an irreducible variety, we have dim(^V) < dim(3V|-i)- As y r U Z r+ \ = y r +i, 
we deduce that 

y r uz r+1 = y r+u 
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which implies by the irrcducibility of y r +i, that Z r +i = y r +\. Consequently, for 
fx < r 2 < R$, wc have 

z ri = y ri £ z r . 2 = y r2 £ % = y^ = s k (c n ). 

Let us prove now that if Rs < r$, we have Z r3 £ S fe (C"). Suppose that Z T3 = S fe (C"), 
then Z T3 is dense in S (C n ) as well as Z-^ in the Zariski topology. This implies that 
Z r3 H Z^ ^ 0, which is false because a tensor cannot have two different ranks. 
Consequently, we have Z T3 £ S fc (C n ). □ 

Proposition 6.5. Ifl<r< Rs, then Z r ^H r . 

Proof. Let r > 1 and A G Z r . Then by definition of 3Vj there exists ^4o G 3^r-i 
and Ai e 3^1 such that A = Aq + A\. As ^4o ^ 3^r-2 (otherwise A € X— l) we have 
A e Z r -i. For £ ^ 0, define A £ = A + eA\. We have that A e G Z r , for all e ^ 0, 
and lim e _,.o = Aq. This shows that Aq € Z r — Z r , and consequently that Z r ^ Z r . 
□ 

The above proposition is about the set of symmetric tensors of symmetric rank 
exactly r. But what about those of symmetric rank at most rl While J^i is closed as 
a dctcrminantal variety, we will sec from Examples 16.61 and 16 . 71 as well as Proposition 
16.81 that y r is generally not closed for r > 1. This is another major difference from 
matrices, for which all y r are closed sets. 

Example 6.6. In dimension n >2, and for any order k > 2, y^ is not closed. 
In fact, take two independent vectors X, and Xj and define the sequence of symmetric 
tensors 

Mi,j)-'=l[<Xi + e*j) 9k -*?"]■ (6-1) 

For any e ^ 0, A s (i,j) is of symmetric rank 2, but converges in the limit as e — > 
to a symmetric tensor of symmetric rank k. In fact, the limiting symmetric tensor is 
easily seen to be a sum of k rank-l tensors, 

Xj ® Xj ® • ■ ■ ® Xj + Xj ® Xj ® • • ■ g) Xj H + Xj ® x.j ® • ■ ■ ® x,-, 

which has symmetric rank k by Provosition \5.6l 

Example 6.7. Let n = 3 and fc = 3. TTien 3^5 C J3, whereas 3 < i?s- /aci, 
take the symmetric tensor associated with the ternary cubic p(x,y,z) = x 2 y — xz 2 . 
According to \16[ this tensor has rank 5. On the other hand, it is the limit of 
the sequence p e {x, y, z) = x 2 y — xz 2 + ez 3 as e tends to zero. According to a result in 
tl61j , the latter polynomial is associated with a rank-3 tensor since the determinant of 
its Hessian is equal to 8x 2 (x — 3ez) and hence contains two distinct linear forms as 
long as e 7^ 0. 

It is easy to show that this lack of closeness extends in general to r > Rs or for 
r < n, as stated in the two propositions below. 

Proposition 6.8. J/Ss < r, then for all k > 2, y r ^ y r . 

Proof. If Rs < r, then y^ £ y r . By the definition of generic symmetric rank, 

y~x s = s k {c n ) = y r . Hence y r £ y r = S k (C n ). Q 

Proposition 6.9. If 1 < r < n, then for any k > 2, y r ^ y r . 

Proof. Take n linearly independent vectors Xi,...,x n . Then the symmetric 
tensors xf k , . . . , x® fe are linearly independent as well, and J2i=i x f k ^ s °f symmetric 
rank r for every r < n by Lemma 15.11 Now for r > 2 and any e / ; define the 
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symmetric tensor 

A £ = I [(xx + ex 2 )® fc - xf fe ] + ^" =3 x® fe . 

A £ is again of symmetric rank r for every e ^ 0, but tends to a symmetric rank r + 1 
tensor (see also Section ISTTj) . For r = 2. the same reasoning applies with 

A t = i[(xi+ex a )®*-xf*]. 

This shows that 3^- is not closed. □ 

Based on these two propositions, we conjecture the stronger statement that for 
order k > 2, the set of symmetric tensors of symmetric rank at most r is never closed, 
even for r = n + 1 , . . . , Rs — 1 . 

Conjecture 6.10. Assume k > 2 and n>2. Then y r ^ y r for any r such that 
1 < r < R s . 

Up to this point, our study has been based on the Zariski topology [48j [T8] . How- 
ever it is useful from a practical point of view to be able to apply these results to 
other topologies, for example, the Euclidean topology. Since the JVs are parameter- 
ized and are thus algebraic constructible sets [48] , and since the closure of an algebraic 
constructible set for the Euclidean topology and the Zariski topology are the same, 
the results in this paper holds true for many other topologies. We have in particular 
the following result. 

COROLLARY 6.11. Let /j, be a measure on Borel subsets ofS (C™) with respect to 
the Euclidean topology on S k (C n ). Let Rs be the generic symmetric rank in S fe (C"). 
If fx is absolutely continuous with respect to the Lebesgue measure on S k (C n ), then 

fx({A € S fc (C") | ranks(A) ? R s }) = 0. 

In particular, this corollary tells us that Z-^ is also dense in S fe (C") with respect to 
the Euclidean topology. It also tells us that the rank of a tensor whose entries are 
drawn randomly according to an absolutely continuous distribution (e.g. Gaussian) is 
Rs with probability 1. This is useful in signal processing for instance, where cumulant 
tensors are estimated from actual data, and are asymptotically Gaussian distributed 

mm- 

These statements extend previous results [3], and prove that there can be only 
one subset Z r of non-empty interior, and that the latter is dense in S fe (C n ); this 
result, however, requires that we work over an algebraically closed field such as C. 

The results of this section are indeed not generally valid over R. We refer the 
reader to Section [H] for further discussions concerning the real held. 

7. Values of the generic symmetric rank. In practice, it would be useful to 
be able to compute the symmetric rank of any given symmetric tensor, or at least 
to know the maximal values of the symmetric rank, given its order and dimensions. 
Unfortunately, these questions are far from resolved. 

The corresponding problem for the generic values of the symmetric rank, however, 
has seen enormous progress due to the work of Alexander and Hirschowitz described in 
Section [73] In fact, even before their breakthrough, bounds on the generic symmetric 
rank have been known for decades [3l [46j [47] : 
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It is known that the lower bound is often accurate but the upper bound is not tight 
[16) . Furthermore, exact results arc known in the case of binary qualities (n = 2) and 
ternary cubics (k = 3) [H [H H3 [35] . 

7.1. Alexander-Hirschowitz Theorem. It was not until the work pQ of Alexan- 
der and Hirschowitz in 1995 that the generic symmetric rank problem was completely 
settled. Nevertheless, the relevance of their result has remained largely unknown in 
the applied and computational mathematics communities. One reason is that the 
connection between our problem and the interpolating polynomials discussed in [T] 
is not at all well-known in the aforementioned circles. So for the convenience of our 
readers, we will state the result of Alexander and Hirschowitz in the context of the 
symmetric outer product decomposition below. 

Theorem 7.1 (Alexander-Hirschowitz). For k > 2, the generic symmetric rank 
of an order-k symmetric tensor of dimension n over C is always equal to the lower 
bound 



Rs{k, n) 



1 (n + k - 1 
n \ k 



(7.1) 



except for the following cases: (fc, n) € {(3, 5), (4, 3), (4, 4), (4, 5)}, where it should be 
increased by 1. 

This theorem is extremely complicated to prove, and the interested reader should 
refer to the two papers of Alexander and Hirschowitz [H [2] . Simplifications to this 
proof have also been recently proposed in fT2] . It is worth noting that these results 
have been proved in terms of multivariate polynomials and interpolation theory, and 
not in terms of symmetric tensors. The exception (fc, n) = (4, 3) has been known since 
1860; in fact, Sylvester referred to it as Clebsh Theorem in his work [52]. It is not 
hard to guess the formula in (17. 1|) by a degrees-of-freedom argument. The difficulty 
of proving Theorem 17.11 lies in establishing the fact that the four given exceptions to 
the expected formula (|7.1j) are the only ones. Table 17.11 below lists a few values of the 
generic symmetric rank. 
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15 


21 
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21 


42 


77 


132 


215 


334 


501 



_Table 7.1 

Values of the generic symmetric rank R$(k,n) for various orders k and dimensions n. 
appearing in bold are the exceptions outlined by the Alexander-Hirschowitz Theorem. 
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Table 7.2 



Generic dimension F(k,n) of the fiber of solutions. 
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7.2. Uniqueness. Besides the exceptions pointed out in Theorem 17. 11 the num- 
ber of solutions for the symmetric outer product decomposition has to be finite if the 
rank r is smaller than or equal to This occurs for instance for all cases of 

degree k — 5 in Table [7TT1 except for n — 5 and n = 10. Hence we may deduce the 
following: 

Corollary 7.2. Suppose (k,n) & {(3, 5), (4, 3), (4, 4), (4, 5)}. Let A e S fc (C") 
be a generic element and let the symmetric outer product decomposition of A be 

Then (|7.2p has a finite number of solutions if and only if 

1 fn + k - 1\ 

Actually, one may easily check the generic dimension of the fiber of solutions by 
computing the number of remaining free parameters [16j : 

F(k,n) = nRs(k,n) — 

This is summarized in Table 17T21 When the dimension of the fiber is non-zero, there 
are infinitely many symmetric outer product decompositions. 

Our technique is different from the reduction to simplicity proposed by ten Berge 
et al. [55l[58], but also relies on the calculation of dimensionality. 

8. Examples. We will present a few examples to illustrate our discussions in 
the previous sections. 

8.1. Lack of closeness. It has been shown [THJ [35] that symmetric tensors 
of order 3 and dimension 3 have a generic rank i?s(3,3) = 4 and a maximal rank 
i?s(3,3) = 5. From the results of Section [6) this means that only Z^ is dense in 
3^4 =3^5, and that Z3 and Z§ are not closed by Proposition 16.51 On the other hand, 
Z\ is closed. 

In order to make this statement even more explicit, let us now define a sequence 
of symmetric tensors, each of symmetric rank 2, that converges to a symmetric tensor 
of symmetric rank 3. This will be a simple demonstration of the lack of closure of 
y r for r > 1 and k > 2, already stated in Proposition 16.81 For this purpose, let 
x, y be two non-collincar vectors. Then the following order-3 symmetric tensor is of 
symmetric rank 2 for any scalar e ^ 0: 

A e = e 2 (x + s^y)® 3 + e 2 (x - s^y)® 3 (8.1) 

and it converges, as e — > 0, to the following symmetric tensor: 

A = 2(x<K>y<g>y + y<£>x<K>y + y®y<8>x). 

This limiting symmetric tensor is of symmetric rank 3. In fact, one may show [14] 
that it admits the following symmetric outer product decomposition: 

A = (x + y r 3 -(x- y r-2y^. 

Now let Xj,yj be linearly independent vectors. 



k 
k 
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By adding two terms of the form (|8.ip , a similar example can be given in dimension 
n . — 4, where we get a sequence of symmetric tensors of symmetric rank 4 converging 
to a limit of symmetric rank 6. 

We will give two more illustrations of Coniecture 16.101 

Example 8.1. If the dimension is n = 3, we can take three linearly independent 
vectors, say x, y, and z. Then the sequence of symmetric tensors A e + z® 3 is of 
symmetric rank 3 and converges towards a symmetric rank-4 tensor. 

In dimension 3, it is somewhat more tricky to build a sequence converging towards 
a symmetric tensor of symmetric rank 5. Note that 5 is the maximal rank for k = 3 
and n = 3. 

Example 8.2. Consider the sequence below as e tends to zero: 

- f(x + ey) m - x® 3 + (z + ex)® 3 - z® 3 l . (8.2) 

£ 

It converges to the following symmetric tensor, which we expressed as a sum of six 
( asymmetric ) rank-1 terms, 

x<g)x<g)y + x®y®x + y<g)x<g>x + z®z®x + z®x®z + X!g)Z!g)z. 

This has symmetric rank 5 since it can be associated with quantic x 2 y + xz 2 , which is 
the sum of (at least) five cubes. 

In terms of algebraic geometry, this example admits a simple geometric interpre- 
tation. The limiting tensor is the sum of a point in the tangent space to 3^1 at x® 3 
and a point in the tangent space to 3^1 at z® 3 . 

Note that the same kind of example can be constructed in the asymmetric case: 

xi <g) x 2 <g> (x 3 - e _1 y3) + (xi + eyi) <g> (x 2 + ey 2 ) <S> £ _1 y3- 

Further discussions of the lack of closeness of y r and the ill-posedness of the best 
rank-r approximation problem in the asymmetric case can be found in |21| . 

8.2. Symmetric outer product decomposition over the real field. We 

now turn our attention to real symmetric tensors. We are interested in the symmetric 
outer product decomposition of A € S fc (M™) over R, i.e. 

A = ^ Aw <g> Vi <g> ■ ■ • <g> Vj (8.3) 

where € K and € K™ for all i = 1, . . . , r. First note that unlike the decomposition 
over C in Lemma H21 we can no longer drop the coefficients Ai, . . . , A r in (|8.3| since 
the fcth roots of Ai may not exist in M. 

Since S fe (R") C S fe (C"), we may regard A as an element of S fe (C") and seek its 
symmetric outer product decomposition over C. It is easy to see that we will generally 
need more terms in (|8.3j) to decompose A over R than over C and so 

rank s , c (^) < rank s , R (A). (8.4) 

This inequality also holds true for the outer product rank of asymmetric tensors. For 
k = 2, i.e. matrices, we always have equality in (|8.4j) but we will see in the examples 
below that strict inequality can occur when k > 2. 
Example 8.3. Let A g S 3 (R 2 ) be defined by 



A = 



' -1 








1 " 
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It is of symmetric rank 3 over 
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whereas it is of symmetric rank 2 over C: 
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where j := 



Hence we see that rankle (^4) 7^ ranks,^^)- 

These decompositions may be obtained using the algorithm described in [15], for 
instance. Alternatively, this tensor is associated with the homogeneous polynomial in 
two variables p(x, y) = Zxy 2 — x 3 , which can be decomposed over R into 

p(x,y) = ^(x + y) 3 + ^(x-y) 3 -2x 3 . 

In the case of 2 x 2 x 2 symmetric tensors, or equivalcntly in the case of binary 
cubics, the symmetric outer product decomposition can always be computed [16j . 
Hence, the symmetric rank of any symmetric tensor can be calculated, even over R. 
In this case, it can be shown that the generic symmetric rank over C is 2 whereas 
there are two typical symmetric ranks over R, which are 2 and 3 

In fact, in the 2x2x2 case, there are two 2x2 matrix slices, that we can call A and 
A\. Since the generic symmetric rank over C is 2, the outer product decomposition 
is obtained via the eigenvalue decomposition of the matrix pencil (Aq,Ai), which 
generically exists and whose eigenvalues are those of AqA^ 1 . By generating (four) 
independent real Gaussian entries with zero mean and unit variance, it can be easily 
checked out with a simple computer simulation that one gets real eigenvalues in 52% 
of the cases. This means that the real symmetric rank is 3 in 48% of the remaining 
cases. This is the simplest example demonstrating that a generic symmetric rank can 
be lacking over R. So the concept of typical symmetric rank is essential to studying 
symmetric tensors over R. 

For asymmetric tensors, the same kind of computer simulation would yield (by 
generating 8 independent real Gaussian entries) typical ranks of 2 and 3, 78% and 22% 
of the time, respectively, leading to the same qualitative conclusions. This procedure 
is not new [531 pp. 13] and has already been proposed in the past to illustrate the 
existence of several typical ranks for asymmetric tensors |37l I55| . An interesting result 
obtained by ten Berge [55J is that pxpx2 real asymmetric tensors have typical ranks 

The problems pertaining to rank and decompositions of real symmetric tensors 
have not received as much attention as their complex counterparts. However, a mod- 
erate amount of work has been done [37J S3 HS1 EH EZ] and we refer the reader to 
these for further information. 

8.3. Open questions. Most of the results that we have presented so far are 
limited to symmetric tensors over the complex field. The case of general asymmetric 
tensors is currently being addressed with the same kind of approach. As pointed 
out earlier, decompositions over the real field are more complicated to handle with 
algebraic geometric tools. In addition, while the problem of determining the generic 
symmetric rank has been resolved thanks to the Alcxandcr-Hirschowitz Theorem, the 
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maximal symmetric rank is known only for particular values of order and dimen- 
sions (e.g. dimension 2); only very rough upper bounds are known for general values. 
Lastly, the computation of an explicit symmetric outer product decomposition for a 
symmetric tensor is computationally expensive, and the conditions (dimension, order) 
under which this can be executed within a polynomial time are not yet clearly known. 
These are problems that we hope will be addressed in future work, cither by ourselves 
or interested readers. 
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